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ABSTRACT 



Predictors of novice learning in simulation 



environments were investigated xn the domain of statistics. The ::irst 
objective was to clarify the relation between intellectual ability 
and working methoa (e.g., orientation and systematical orderliness) ^ 
and to determine the effects on learning of working meth^xl^ 
independently of intellectual ability, A second objective was tc 
determine whether instructional aid by presenting students with a 
well- structured task, :^.istead of unguided learning by discovery, 
might compensate for lack of ability or a poor working method* 
Twenty--seven first-year university students of relatively high 
intelligence (n=i4) or low intelligence (n=13) worked in either a 
structured or an unstructured learning environment. Thinking-aloud 
protocols were analyzed in terms of the quality of working method. 
Results indicate that the working method of high-intelligence 
subjects was significantly better than that of subjects of low 
intelligence; working method was also a strong predictor of learning, 
independent of intellectual ability. No learning effects due to the 
level of structure of the learning environment could be detected. 
Five data tables and a 28^item list of references are included. An 
appendix contains three definitions from the study, (Author/SLD) 
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ABSTRACT 



Predictors of novice learning in simulation environments were investigated in the 
domain of statistics. The first objective was to clarify the relation between intellectual 
ability and working method (e g. orientation and systematical orderliness), and to 
determine the effect on learning of working method, independent of intellectual ability. A 
second objective was to determine whether instructional aid by presenting students with a 
well-structured task, instead of unguided learning by discovery, might compensate for 
lack of ability or a poor working method. Twenty-seven relatively high or low intelligent 
first year university stut'^iits worked in either a structured or an unstructured learning 
environment. Thinking-aloud protocols were analysed on the quality of working method. 
The results indicated that the working method of high intelligent subjects was 
significantly better than that of low intelligent subjects, but that working method also is a 
strong predictor of learning, independent of intellectual ability. No learning effects due to 
stnicturedness of learning environment could be detected. 
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The novice-expert paradigm has yielded a vast amount of research on differences 
between novice and expert problem solving (Laridn, McDeimott, Simon & Simon* 19^; 
Chi, Feltovich & Glaser, 1981; Schoenfeld & Herrmann, 1982; Anderson, 1985; Chi, 
Glaser & Farr, 1988; Thibodeau Hardiman, Dufresne & Mestrc. 1989). Besides 
quantitative differences (e.g. number of errors and time on task), a number of qualitative 
differences have been found. Experts not only have more knowledge at their disposal, but 
their knowledge is also better organized (Chi, Glaser & Rccs, 1982) and mwc of a 
piwcdural kind (Anderson, 1985; Jansweijer, Elshout & Wielinga, 1990). Before 
actually acting, experts pass through an elaborate qualitative analysis of the problem 
through which schemata of forward problem solving strategies are activated. Novices on 
the other hand, analyse a given problem in terms of superficial features (Chi et al., 1982) 
and arc inclined to act immediately and unsystematically (Elshout, 1987; Jansweijer et al., 
1990). This lack of metacogniiive control results in a poor problem representation to 
which only weak problem solving strategies like means-ends analysis and working 
backwards can be applied (Glaser & Bassok, 1989; Glaser, 1990). 

Not all novices are limited to equally poor problem solving behavior (Schoenfeld & 
Herrmann, 1982). Contrary to genuine novices, an 'expen-novice* tends to act more 
expert-like by orientating on a problem, working more systematically and more 
accurately, and by evaluating more during the problem solving process (Elshout, 1985). 
Furthermore, Chi et al (1989) demonstrated that better students elaborau-d more on the 
subject matter by generating self-explanations during the learning process. Deep 
orientation, systematical orderliness, evaluation and elaboration are the characteristics of a 
pit)ficient working method. Such an effective working method is a metacognitive skill, or 
in terms of Glaser (1990) a self-regulatory ability, which is brought in by the student in 
order to structure the learning process (Veenman, Elshout & Bierman, 1989b). 
Metaphorically spoken, an effective working method might be regarded as a student's 
compass and skeleton-map on a voyage of discovery in a new domain. Without such an 
adequate outfit a novice expedition is likely to run ashore, especially for low intelligent 
students (Veenman, Elshout & Bierman, 1991). 

There are three models for describing the relation between intellectual ability and 
working method. The first model regards an effective working method as a manifestation 
of intellectual ability. According to this model, working method cannot have a predictive 
value for learning, independent of intellectual ability. In a second, contrasting nnodel 
working method and intellectual ability are independent predictors of learning. The last 
model thus is a mixed model, in which working method has a surplus value on top of 
intellectual ability, for the prediction of learning, while high intelligent novices tend to 
exhibit a better working method. 

Empirical studies do not consistently support one of these nwdels. Veenman and 
Elshout (Veenman et al., 1989b; Elshout & Veenman, 1990) reported the results of an 
experiment in the domain of heat theory, that corroborated the furst model. They had high 
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and low inteltigent subjects working for about an hour in a computer emulated Heat Lab 
while thinking aloud. Tape-recorded ihinking-aloud protocols were analysed on the 
quality of working method. High intelligent subjects showed both a more proficient 
woriting method and a significantly better learning performance than low intelligent 
subjects. Working method correlated significantly with learning measures, but when 
intelligence was paitialed out these (»)iTelations approximated zero. Some suppoft fc»- the 
second or the third model was gathered by Swanson (1990), though the research was 
primarily concerned with problem solving instead of learning. Children, who were 
selected on high or low aptitude (as measured by a cognitive abilities test) and on high or 
low metacogniuve knowledge (by means of a questionnaire), had to perform two 
Piagetian tasks. Swanson's results showed that high-metacognitive individuals 
outperformed low-metacognitive individuals regardless of their aptitude level, indicating 
that metacognitivc skills can compenwte for cognitive ability. Swanson's experinjental 
design, which forces intelligence and metacognitivc knowledge to be orthogonal, does 
not permit to decide whether the second or the third of the three nnxiels is favored by 
these results. Recent research in the domain of electricity (Vecnman & Elshout, 1990) 
however, provided support for the mixed model. High and low intelligent subjects 
worked for several hours in a simulated Electricity Lab, learning the principles of Ohm's 
law. The results indicated that, while working method and intellectual ability correlated 
moderately, working method contributed significantly to the learning process, 
independent of intellectual ability. Clearly, the relation of working method with 
intellectual ability is an intricate one that remains to be clarified. 

Realistic learning environments involve learning to solve complex and knowledge- 
rich problems which are representative for everyday problem solving. Simulations, used 
as an instructional tool, allow for learning by doing and learning by discovery under 
restricted realistic conditions. They are said to enable students to bridge the gap between 
reality and abstract knowledge, to improve motivation and enhance learning by an active 
student interaction, and in general to provide students with an appropriate cogiitivc and 
affective context for learning (de Jong, 1991). Essential for simulation as instructional 
tool is the continuous presence of learner activities. Students need to engage in exploring 
a domain thoroughly by generating hypotheses and testing these hypotheses by actively 
designing and performing experiments in the simulation environment (Reimann, 1989; 
Goodyear, Njoo, Hijne & van Berkum, 1990). Because of its explorative nature, learning 
with simulations involves complex problem solving, inductive reasoning and discovery 
learning, which puts a high cognitive demand on the student (Goodyear ct al., 1990; de 
Jong, 1991 ). A large number of alternative actions might overwhelm students and result 
in a trial-and-error mode of behavior (van Berkum & de Jong, 1990). On the other hand, 
students might also adopt a waiting attitude, not using the opportunities for learning that 
the simulation environment offers (Njoo & de Jong, 1989) or not being able to use them 
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pro}>erly due to limited intellectual resources or a poor working method (Veenman et al., 
1991). It is generally conceived that a completely learner controlled situation is 
problematic, especially for the weaker students (Goodyear et al., 1990). As a 
consequence, de Jong and others (van Bcrkum & de Jong, 1990; de Jong, 1991) take the 
position that learning in simulation environments should be accompanied with at least 
some instructional guidance. 

Instructional aid by presenting students with a well-structured simulation 
environment, might compensate for lack of ability or a poor working method. The 
conceptual difficulties of low intelligent students, due to the complexity of the subject 
matter, could be relieved by piestnicturing the information that is to be processed. By 
providing them with pre-set experiments that elaborate on essentials of the domain, 
students should be urged to formulate and evaluate hypotheses, to vary the values of 
variables systematically, and to reconsider possible misunderstandings (Collins & 
Stevens, 1983). Instructional aid though, should not be limited to non-specific 
interventions (e.g. passively providing students with pre-set experiments or simply 
structuring the subject matter without specific feedback), since in several studies these 
interventions did not improve learning (Veenman et al., 1989b; Elshout & Veenman, 
1990; Veenman & Elshout, 1990). Even when guided by instruction, the learning 
activities of low intelligent students invariably needed explicit corrections (Veenman et 
al., 1991). Consequently, low intelligent students, who mipht also suffer from a poor 
working method, are expected to profit more from a structured learning environment in 
which explicit f^back is included. 

In a new experiment, which was des'gned in another domain to avoid domain 
specific effects, the concept of correlation waf, focussed on as an appropriate study-theme 
for our subjects. The correlation between two dichotomic variables is a topic with 
sufficient complexity to bring forth an elaborate learning process, but still relates easily to 
everyday experiences (Inhelder & Piaget, 1958). In discovering principles of correlation 
by experimenting with data-sets, intellecnial ability and working method are likely to play 
.in important role. Furthermore, instructional guidance combined with extensive domain- 
specific feedback is expected to steer the learning process, which might be profitable to 
particularly low intelligent students. 

METHOD 

Subjects. 

Some months prior to the experiment intellectual ability was assessed by a series of ability 
tests, representmg several components of the Structure-of- Intellect HKxiel (Guilford, 
1967). Those first-year psychology students whose composite scores deviated at least 
one standard deviation from the mean (M= 17.47, sd= 3.88, N= 459), were denominated 
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as either high or low intelligent. Fourteen high and thirteen low intelligent stiuients who 
recently had followed a introductory statistics course, participated in the experiment 
Though all of them were familiar with some general principles of correlation, they were 
ignorant of the phi-coefficient 

Learning conditions. 

A statistics environment was implemented in Course of Action, an authoring language for 
Macintosh computers (Vecnman, Balk & Biemnan, 1989a). Data-sets, d^ved from 
eveiyday examples, could be altered by the student and the cwrelation could be calculated 
by the program on request. By estimating correlations, stiulents first had to determine 
which values contributed positively to the correlation and which values contributed 
negatively. Next, they had to discover a simple method fw inclusion of both positive and 
negative values into a correlational measure: the Piaget fwmula (adapted fhwn Inhelder & 
Piaget, 1958). This formula, based on a simple additive model, disregards the skewness 
of distributions. Finally, students had the opportunity to experience the effects of 
distributional skewness by determining the constraints for application of the Piaget 
formula. They were presented subsequently with a general formula, the phi-coefficient 
which is based on a nwre complex, multiplicative nxxlel. 

Simulation environments were implemented separately for a structured learning 
condition with instructional guidance and for an unstructured condition that allowed for 
unguided discovery learning. The data-sets in the structured simulation environment were 
presented in two by two diagrams throughout the course, while students in the 
unstructured environment had to reorganize the data by themselves. Furthermore, a 
computer assisted instruction shell overiaid the structured simulation environment. This 
CAI shell served as a tutor for teaching and explaining parts of the subject matter, 
provided students with guidance during experimentation, and offered students the 
opportunity to jump backwards for consulting previously given information. 
Accordingly, the construction and interpretation of two by two diagrams was explicitly 
taught in the structured environment, while students in the unsirucnired condition only 
received a short explanation about the series of data displayed. During estimation of 
correlations, students in the structured environment received specific feedback if their 
estimation was not proximate. This feedback included a second diagram constructed from 
the estimated correlation, that was to be compared with the original diagram and its true 
correlation. E)uring experimentation, students in the stnK^tured condition were required to 
generate a hypothetical formula and to ento* a value calculated by that formula, before the 
true correlation for a certain diagram was revealed to them. General feedback was given 
about the extent the calculated conelation deviated from the trae conelation. Howevw, the 
entered values were also matched by the program to some singular respons patterns (for 
instance the tendency to neglect the data that contribute to the correlation negatively) and 
specific feedback was added to these responses. Furthermore, students in the structured 
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condition received explicit instructions about how to perform a number of 'telling' 
experiments. An example of such an instruction was to make one cell of the diagram zero 
and then, step by step, raise the value of that cell with one point whilst lowering the other 
one on the diagonal of the diagram (thus keeping the value of Piagei's formula constant 
and changing the phi-coefficient systematically; see the appendix). Students in the 
unstructured condition on the other hand, had to design their own experiments and 
received no feedback. While formulas in the unstructured condition were presented 
without further comment, they were explained thoroughly in the structured condition with 
questions addressing students on their understanding of the subject matter. Actually, the 
structured condition provided students with extensive support during the learning process 
in the simulated environment. 

Procedure. 

The subjects were assigned randomly to either the structured or the unstructured learning 
condition. The unstructured condition contained 6 low and 7 high intelligent subjects, 
while 7 low and 7 high intelligent subjects were engaged in the structured condition. 
Before entering one of the learning environments, all subjects received a short instruction 
about operating the Macintosh computer and a general refreshment course in basic 
statistics. During their work in the simulation environment, which took about 2 to 3 
hours, thinking-aloud protocols of all subjects were tape-recorded. Time on task was not 
controlled, but was included in the study as a concomitant variable. Student notes, if any, 
were preserved in order to be analysed in relation to the protocols. The protocols were 
scored for quality of working method (the five scales being Orientation Activities, 
Systematical Orderliness, Accuracy, Elaboration and Evaluation Activities) and the level 
of learning reached. The analyses were conducted by two judges who received no prior 
information about the student's intelligence-test scores. They performed the analyses 
together, arguing until agreement was reached. 

Aspects of the Working Method were scored separately from the protocol segments 
corresponding to three subdivisions in the learning program: learning to estimate 
correlations, discovering principles of the Piaget formula and exploring constraints for 
using the Piaget formula. These protocol segments appeared to have the substantial length 
which is required for protocol analysis. (Quality of Orientation Activities was judged on 
indications of analysing a problem, building a mental model of the task and generating 
predictions. Judgements of Systematical Orderiiness were based on the quality of 
planning and systematical execution of those plans. Criteria for Accuracy were precision 
of calculation, correct usage of quantities and the avoidance of negligent mistakes. 
Evaluation Activities were judged on monitoring and checking, while judgements of 
Elaboration were concerned with generating explanations, relating the subject matter and 
recapitulating. All aspects were rated on a ten-point scale and average scores on the five 
sub-scales were computed fw each subject. 
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The level of learning reached, was scored from the thinking-aloud protocols by 
subdividing the learning program in coherent parts and rating the (^iresponding protocol 
segments on a ten-point scale. Five parts were disriminatcd and scored on mastery of the 
specific subject matter: construction of a ctiagram or table, estimation of correlations, 
learning the principles of the Piaget formula, learning the distributional effects on the 
Piaget formula and calculation of the correlation coefficient. For each subject an average 
score was calculated. 

Problems. 

In a pretest-posttest design subjects were presented with a series of problems prior to and 
shortly after the experimental course. After a delay of three weeks the subjects were 
offered another series of problems as a retention test. The problems dealt explicitly or 
implicitly with the correlation between two dichotomous variables in everyday siniations. 
Explicit problems actually asked subjects to 'calculate the correlation before drawing a 
conclusion. Implicit problems on the other hand, did not ask for a calculation but 
requested a justified conclusion. Hence, the explicit problems measured the acquired 
knowleoge and competency in correlational calculation, while the implicit problems 
additionally measured transfer of learning. Solving the problems of both types required a 
fair amount of qualitative as well as quantitative reasoning about correlation. An implicit 
problem was for instance: 

In research of the relation between job-responsibility a*id stress-experience 
sixty subjects are included, of whijh 30 percent is having a responsible job. 
A stress-questionnaire shows thru 60 percent of all subjects are experiencing 
job-related stress, while eight subjects with a responsible job are experiencing 
stress. What is your conclusion about the relation between job-responsibility 
and stress-experience? Explain your answer. 

The pretest consisted of only 4 implicit problems. Both the posttest as well as the 
retention test were composed of 4 implicit and 5 explicit problems, which were presented 
in a jumbled sequence. The 4 implicit problems corresponding to the pretest items, will be 
denominated as respectively posttest 1 and retention test 1. The 5 problems with explicit 
instruction will be referred to as posttest 2 and retention test 2. 

Thinking-aloud protocols of the pretest, posttest and retention test problems were 
analysed prior to the course-protocols by the two 'blind' judges. The problems were 
scored one at the time for all subjects. Each problem was rated as a school mark on a ten- 
point scale, as is usual in Dutch education (with 5 being the highest fail-score and 6 being 
the lowest pass-score). The judgements were based on three criteria for solving the 
problem correctly: converting the problem statement to a ctarect diagram or table, (partial) 
calculation of the correlation coefficient and drawing a justified conclusion. A small 
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miscalculation was not severely penalized. For each subject an average score on the 
pretest, posttest 1 , posttest 2, retention test 1 and retention test 2 was calculated. 

RESULTS 

Problems. 

The internal consistency of the pretest was rather low (alpha was .39), presumably 
indicating that the subjects have only slight initial knowledge of two by two contingency 
relations. Though, in accordance with this interpretation, the general level of pretest 
scores was poor, an Anova showed a significant effect of intellectual ability with 
F(l,23)=8.00 (p<.01), indicating that high intelligent subjects possessed slightly more 
knowledge about correlations to stan with. The correlation between intelligence and 
pretest scores was 50 (.36 after correction for extreme groups of intellectual ability, 
following the procedures of Gulliksen (i%l )). 

The reliabilities of the posttest were .84 and .83 for respectively posttest 1 and 
posttest 2. Anova's on both posttests showed robust intelligence effects (see Table 1) 
with high intell'gent subjects performing better than low intelligent subjects. No effects 
due to learning condition or interaction were obtained. 

The reliabilities of retention test 1 and 2 were respectively .86 and .87. Strong 
intelligence effects were established with Anova's on both retention tests (see Table 1). 
Again, no learning condition effects were obtained for both tests, but the interaction effect 
was significant for retention test 2 (see Table 1). However, this interaction effect certainly 
did not confirm the hypothesis that low intelligent subjects would profit more from the 
structured learning environment compared to high intelligent subjects. The scores of low 
intelligent subjects in the unstructured condition actually surpassed those of low 
intelligent subjects in the structured condition (see Table 3), 

Tabic 1. Results of Anova's on Uic learning measures. 



Learning IQ Learning condition Interactiai 

mca.sures. — — — — — 

Posucsl 1 22.52 (p<.0001) 0.01 0.00 

Poslicsi 2 25.36 (p<.0001) 0.53 0.05 

Ret. ie.si 1 # 12.60 (p=.0O18) 0.09 0.31 

Ret. test 2 # 47.51 (p<.0001) 0.29 6.56 (p<.02) 

Level ofleaming 17.06 <p=.0004) 1.35 1.35 



Except for # wiUi F(l .22). elsewhere F(U3) is applicable. 
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An Anova with repeated measures on the implicit problems (pretest, posttest 1 and 
retention test 1) with intellectual ability as the between-persons source of variance and 
time of testing as the within-persons f&cxor, evidently showed a signiHcaiit effect of 
intellectual ability with F(l,24)=21.29 (p<.U001). Furthennore, significant effiects were 
obtained for time of testing (F(2,48)=4.34, p<.02) and the interaction of intellectual 
ability with time of testing (F(2,48)=6.58» p=.003). The mean scores in Table 2 clearly 
indicate that the high intelligent subjects learned more form the computerized course than 
low intelligent subjects, which difference persisted after the three weeks delay. 



Tabic 2. Means and standard devisaions (in parentheses) of the imfritctt piobtenis. 

Pretest Posuesi 1 Retention test 1 
UQ HIQ UQ HIQ UQ HIQ 

Unsuwturcd 4.13 4.79 3.71 5.96 4.35 5.96 

condition. (0.82) (0.51) (1.07) (1.29) (1.68) (1.42) 

Structured 4.00 4.82 3.75 6.01 3.89 6.11 

condition. (0.69) (0.69) (0.76) (1.63) (1.05) (1.33) 



Table 3. Means and .standard deviations (in parentheses) of Posttest and Retention test with explicit 
calculation-instruction. 

Posttest 2 Retention test 2 
UQ HIQ UQ Hlg 



Uastiucturcd 4.73 

condition. (1.3 1) 

Stnictufcd 4.51 

condition. (1.57) 



6.97 4.88 6.23 

(0.58) (0.76) (0.56) 

6.57 3.91 6.86 

(0.66) (1.08) (0.64) 



Level of learning. 

The internal consistency of the protocol- scored level of learning was satisfactory (alpha 
was .75). Correlations with other measures of learning ranged from .57 to .67. An 
Anova on the level of learning scores reveal«l a Mgnificant effect of intellectual ability 
(sec Table 1 ) with the scores of high intelligent subjects exceeding those of low intelligent 
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subjects. No effects of learning condition or interaction were established. Because the 
means in Table 4 might suggest an effect of learning conditions for low intelligent 
subjects in particular, an Anova was performed on their separated scores. However, the 
result of F(M1)=2.15 (p=.17) confirmed the general conclusion that effects due to 
learning condition failed to appear. 



Tabic 4. Means and stamiard dcviaticNis (in parentheses) of Level of learning scores. 

Level of learning 
UQ HIQ 

Uastnicturcd 4.87 6.60 

condition. (0.98) (0.97) 

Stnsruircd 5.63 6.60 

condition. (0.89) (0.48) 



Working Method. 

Alpha reliabilities for the five sub-scales of Working Method were .88, .85, .63, .81 and 
.88 for respectively Orientation Activity, Systematical Orderliness, Accuracy, Evaluation 
Activities and Elaboration. All subjects were real novices, that is to say, they showed 
rather poor Orientation activities. However, a si^ificant effect of intellectual ability with 
F(l,23)=27.56 (p<.0001) indicated that high intelligent subjects orientated more than low 
intelligent subjects. As for Systematical Orderliness, subjects especially differed in the 
systematic implementation of a carefully designed sequence of data manipulations. High 
intelligent subjects were rated significantly higher on Systematical Orderliness than low 
intelligent subjects (F(l,23)=21.34, p<.0001). Though the intelligence effect on 
Accuracy was not significant (F(l,23)=3.65, p<.07). there is a minor suggestion for high 
intelligent subjects to work somewhat more accurately than low intelligent subjects. For 
Evaluation Activities also an intelligence effect in favour of high intelligent subjects was 
established (F(l,23)= 15.76, p=.(K)06). Elaboration, which often took the form of re- 
orientation after feedback was given, showed a similar effect of intellectual ability with 
F(l,23)=19.10 (p=.0002). 

A general Working Method score was composed of all scores on the distinct 
measures together with two additional measures for completeness of diagrams drawn and 
tidiness of student-notes. The internal consistency of this composite score was high 
(alpha was .96). An Anova on Working Method scores showed a strong effect of 
intellectual ability with F(l,23)=21.20 (p<.0001). Working method correlated .68 with 
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intellectual ability (.52 after c(»rection for selection of extreme groups). The correlations 
of Working Method with the learning measures were high and, except for Posttest 1, 
these correlations remained high after intellectual ability was partialed out (see Table S). 
Furthermore, after correcting for selection of extreme groups of intellectual ability, all of 
the partial correlations of Working Method with the learning measures were highly 
significant (see Table S). 



Table S. Results or the conclationai analyses. 



Learning W.M. IQ W.M. IQ W.M. pM.cor. 

measures. pait.cOT. (cMTBoed) (conccted) 



Prcicsi 


0.56 


0.50 •* 




0.36* 




Posiicst 1 


0.65 ** 


0.72 *• 


0.32 


0.56 


0.50** 


Posttesi 2 


0.76 *• 


0.71 *♦ 


0.55 ** 


0.56 *• 


0.67 


Rci. icsi 1 


0.74 •* 


0.66 ** 


0.54 »♦ 


0.50 


0.65 


Ret. ic.si 2 


0.81 *♦ 


0.82 •* 


0.62 *• 


0.69 *• 


0.74 


Level of learn. 


0.85 ** 


0.(A ** 


0.74 •* 


0.49 


0.80 •» 



'W.M'. in<^s woricing mcihod; 'correcled' means correcuxl for sclcciim of extreme ^oupsof IQ; 
• p<.05: ** iK.Ol. 

Time on task. 

The differences in time on task between both learning conditions were considerable 
(F(l,23)=11.88, p<.(X)3). Subjects in the structured condition spent on the average 131 
minutes working through the program, while it took subjects in the unstructured 
condition only 1(X) minutes on the average. However, correlations between time on task 
and learning measures were invariably low (ranging from -.23 to .1 1). 

DISCUSSION 

The results of this study coriirmed once more that learning by doing is mediated by 
both intellectual ability and an effective working method. Furthermore, it was shown that 
high intelligent subjects exhibited an evidently better working method than low intelligent 
subjects. However, working method also proved to be a strong predictor of learning, 
apart from intell^tual ability. These results clearly fit the mixed model, in which high 
intelligent novices tend to have a better working method but in which working method 
has a unique predictive value for learning too. 
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Extensive structuring of the learning environment did not ir *prove learning, nor 
could an Aptitude-Treatnient Interaction between intellectual ability ami structuredncss of 
learning environment be established. These results are in line with previous research of 
Veenman and Elshout (Veenman et al., 1989b; Elshout Jfc Veenman, 1990; Vcenman & 
Elshout, 1990). Though the instructional shell of the structured condition raiginated from 
a CAI tradition, some intelligent features made a considerable anjount of branching 
possible as a consequence of student reactions. However, adding specific feedback to 
guided discov zry learning did not compensate sufficiently for lack of ability or a poor 
working method. One possible reason for the absence of learning effects due to 
stnicturcdness of learning environment emerged from the thinking-aloud protocols. By 
not reading an instruction thoroughly and as a const.uience doing the wrong things, by 
not completing a sequence of tasks and by skipping ihe feedback, some students reduced 
the structured learning environment to an unstructured one. A poor working method may 
have prevented them from taking advantage of the stmcnired learning environment Very 
similar passive attitudes of students were observed by Njoo and de Jong (1989). 

Clearly, most novices lack the prior knowledge, domain-specific knowledge as well 
as knowledge about how to perfomi an experiment, which is considered a prerequisite for 
successful learning in simulation environments by Goodyear et al. (1990). But for low 
inieP'gent novices it is the accumulation of knowledge deficiencies, limited intellectual 
resources and a poor working method that causes them to go astray during their Odyssey 
in a simulated environment. Even instructional guidance and feedback cannot provide 
sufficient help if the conceptual complexity of the task is making too high demands on the 
student. 

Since working method appears tc >iave a unique predictive value to learning, 
metacognitive mediation directed at raising the level of working method, might improve 
novice learning. This mediation is now being built into the electricity lab which was used 
by Veenman and Elshout (1990) and is expected to enhance learning through the 
improvement of working meihod. 
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APPENDIX 



Two bv two cnntinpencv diagram: 



a+b 



c+d 



a+c b+d 



In this two by two contingency diagram the four observations a, b, c and 
d represent the possible combinations o. »wo dichotomous variables: a is positive on both 
variable 1 and variable 2, b is positive on variable 1 and negative on variable 2, c is 
negative on variable 1 and positive on variable 2, and d is negative on both variable 1 and 
variable 2. 



Piaget formula: 



(a+d) - (b+c) 



n 



With n equals (a+b-K:-Kl). This formula, adapted from Inheldcr Piaget (1958), is based 
on a simple additive model and disregards the skewness of distributions. Piaget's formula 
is only applicable if the marginal frequencies (a+b), (c+d), (a-nc) and (b+d) are equal. 
Hence, a precondition for using Piaget's formula is a 50 percent distribution for both 
dichotomous variables. 



Phi coefficient: 



ad- Ix: 



V(a+b)(c-Hl){a+c)(b+d)' 



Calculation with the phi coefficient always results in the same correlational value as 
calculation by the product-moment correlation coefficient would, even if the marginal 
frequencies arc unequal. 
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